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Abstract 

First-order approximation and model management is a 
methodology for a systematic use of variable-fidelity models 
or approximations in optimization. The intent of model man- 
agement is to attain convergence to high-fidelity solutions with 
minimal expense in high-fidelity computations. The savings in 
terms of computationally intensive evaluations depends on the 
ability of the available lower-fidelity model or a suite of models 
to predict the improvement trends for the high-fidelity problem. 
Variable-fidelity models can be represented by data-fitting ap- 
proximations, variable-resolution models, variable-convergence 
models, or variable physical fidelity models. The present 
work considers the use of variable-fidelity physics models. We 
demonstrate the performance of model management on an aero- 
dynamic optimization of a multi-element airfoil designed to 
operate in the transonic regime. Reynolds-averaged Navier- 
Stokes equations represent the high-fidelity model, while the Eu- 
ler equations represent the low-fidelity model. An unstructured 
mesh-based analysis code FUN2D evaluates functions and sen- 
sitivity derivatives for both models. Model management for the 
present demonstration problem yields fivefold savings in terms 
of high-fidelity evaluations compared to optimization done with 
high-fidelity computations alone. 

Key Words: Aerodynamic optimization, airfoil de- 

sign, approximation concepts, approximation manage- 
ment, model management, nonlinear programming, sur- 
rogate optimization, variable-fidelity 

Background 

Approximations and low-fidelity models have long 
been used in engineering design to reduce the cost of opti- 
mization (e.g., [ 1-3]). An overview of approximations in 


structural optimization, for instance, can be found in [4]. 
Accounts of recent efforts in developing methodologies 
for variable-complexity modeling are relayed in [5, 6|. 

The present work concerns an approach, the Approx- 
imation and Model Management Framework (AMMF) 
[7-10], designed to enable rapid and early integration of 
high fidelity nonlinear analyses and experimental results 
into the multidisciplinary optimization process. This is 
accomplished by reducing the frequency of performing 
high-fidelity computations within a single optimization 
procedure. 

Until recently, procedures for the use of variable- 
fidelity models and approximations in design had relied 
on heuristics or engineering intuition. In addition, with 
a few exceptions (e.g., [1 1], 1 12]), the analysis of algo- 
rithms had focused on convergence to a solution of the ap- 
proximate or surrogate problem ([13], [14]). The AMMF 
methodology discussed here and in related papers is dis- 
tinguished by a systematic approach to alternating the use 
of variable-fidelity models that results in procedures that 
are provably globally convergent to critical points or so- 
lutions of the high-fidelity problem. 

Model management can be, in principle, imposed on 
any optimization algorithm and used with any models. In 
[15], we considered AMMF schemes based on three non- 
linear programming methods and demonstrated them on 
a 3D aerodynamic wing optimization problem and a 2D 
airfoil optimization problem. In both cases, Euler analy- 
sis performed on meshes of varying degree of refinement 
formed a suite of variable-resolution models. Results in- 
dicated approximately threefold savings (similar across 
the three schemes) in terms of high-fidelity function eval- 
uations. The AMMF based on the sequential quadratic 
programming (SQP) approach was judged to be the most 
promising for single-discipline problems with a modest 


•Member AIAA. Multidisciplinary Optimization Branch, MS 159. NASA Langley Research Center. Hampton. VA. 23681-2199. 
n . a 1 exandrov@larc . nasa . gov. 

f Member AIAA, Computational Modeling and Simulation Branch. MS 128, NASA Langley Research Center, Hampton. VA. 23681-2199. 
e . j . nielsen@larc . nasa , gov. 

f Member AIAA. ICASE, MS 132C. NASA Langley Research Center, Hampton. VA. 2368l-2199.buckaroo@icase.edu. This author's 
research was supported by the National Aeronautics and Space Administration under NASA Contract No. NAS 1-97046 while the author was in 
residence at the institute for Computer Applications in Science and Engineering (ICASE), NASA Langley Research Center. Hampton. VA 2368 1 . 

^ Member AIAA, Computational Modeling and Simulation Branch. MS 128. NASA Langley Research Center, Hampton. VA, 23681-2199, 

w . k . anderson@larc . nasa . gov. 

Copyright ©2000 by the American Institute of Aeronautics and Astronautics. Inc. No copyright is asserted in the United States under Title 17. U.S. 
Code. The U.S. Government has a royalty-free license to exercise all rights under the copyright claimed herein for Government Purposes. All other 
rights are reserved by the copyright owner. 


1 

American Institute of Aeronautics and Astronautics 



number of design variables, as well as for certain formu- 
lations of the MDO problem. 

The study in [15] has served as a proof of con- 
cept for AMMF in the case where low-fidelity models 
are represented by data-fitting approximations (kriging, 
splines and polynomial response surfaces) or variable- 
resolution models. The present work considers, arguably, 
the most challenging combination of the high and low- 
fidelity models within a single optimization procedure - 
that of variable-fidelity physics models. The performance 
of the first-order model management is demonstrated on 
an aerodynamic optimization of a multi-element airfoil. 
Variable- fidelity models are represented by an unstruc- 
tured mesh-based analysis run in viscous and inviscid 
modes. 

In the next section, we describe the AMMF under in- 
vestigation and discuss the points of interest for the cur- 
rent study. We then present the demonstration problem, 
followed by a discussion of the numerical experiments 
and results. 

AMMF under investigation 

For the present demonstration, the optimal design 
problem is represented by the bound constrained nonlin- 
ear programming problem: 

min /(*) 

s. t. I<j < u, ’ 

where ./* is the vector of design variables, the objective 
/ is a continuously differentiable nonlinear function, and 
I < x < u denotes bound constraints on the design vari- 
ables. 

The first-order AMMF used here for solving (1) is 
based on the trust-region strategy, which is a methodol- 
ogy for the improvement of global behavior of the local 
model-based optimization algorithms 1 16|. The following 
pseudo-code describes the AMMF. 

Initialize .r ( ., A r 

Do until convergence: 

Select a model a c such that 

«, (■*', ) = f(j\ ) and V« r (.(•<) = V/(j-,.) 

Solve approximately for s v ~ .r — x c : 
min o r (.r c + s) 

s. t. / < .r < r/ 

II * Ik < A, 

Compute /><■ = 

Update A ( and x c based on p c 

End do 

Details of the updating strategy can be found, for instance, 
in [7]. Briefly, the point j* t . is accepted if the step s c re- 
sults in a simple decrease in the objective function, i.e., if 


/(jy) > /(*<• 4- -v). Otherwise the step is rejected. The 
fry, trust region (chosen because it interacts naturally with 
the bound constraints) is decreased if p c is small. Experi- 
ence suggests that “small” be taken as less that 10~ : \ If 
p c is close to one or greater than one (this indicates excel- 
lent predictive properties of the model), the trust region is 
doubled. Otherwise, it is left unchanged. 

The conditions 

a c {x r ) = /(.* v) (2) 

Va r (.r c ) = V/(j* c ) (3) 

are known as the first-order consistency conditions, which 
we will discuss presently. 

In conventional optimization, a c is usually a linear or 
quadratic model of the objective /. AMMF replaces this 
local. Taylor series approximation by an arbitrary model 
required to satisfy the consistency conditions (2)-(3). Re- 
gardless of the properties of the low-fidelity model, the 
consistency conditions force it to behave as a first-order 
Taylor series approximation at points where they are sat- 
isfied. Solving the subproblem of minimizing a c is itself 
an iterative procedure that now requires the function and 
derivative information from the low-fidelity model. 

First-order AMMF methods can be shown to converge 
to critical points or solutions of the high-fidelity problem 
under appropriate standard assumptions of continuity and 
boundedness of the constituent functions and derivatives 
(see [9], for instance), given that the consistency condi- 
tions (2 )-(3) are imposed at each major iterate r r . 

Qualitatively, the reason a first-order AMMF con- 
verges to an answer of the high-fidelity problem may be 
summarized as follows. Although a lower- fidelity model 
may not capture a particular feature of the physical phe- 
nomenon to the same degree of accuracy (or at all) as 
its higher-fidelity counterpart, a lower-fidelity model may 
still have satisfactory predictive properties for the pur- 
poses of finding a good direction of improvement for the 
higher-fidelity model. By imposing the consistency con- 
ditions, AMMF ensures that at least at the major iterates, 
the lower-fidelity model provides the same direction of 
descent as the high-fidelity counterpart. Two questions 
arise. Flow easy is it to impose the first-order consistency? 
How does the method perform in practice? 

The answer to the first question is that imposing the 
conditions (2)-(3) is straightforward using a correction 
technique due to Chang et al. 1 1 7] This technique cor- 
rects a low-fidelity version j) 0 of an arbitrary function so 
that it agrees to first-order with a given high-fidelity ver- 
sion f } u . This is done by defining the correction factor ii 
as 
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Figure I : Mesh for the viscous model 



Given the current design variable vector one builds a 
first-order model ji c of 3 about j\ : 

l3 r [x) = iHx r ) + Vftx e ) T [*-x e ). 

The local model of 3 is then used to correct //„ to obtain 
a better approximation «(,r) of //,*: 

fhi(i') = % <i(j ) = l1 c (r)fio { •«•)• 

The corrected approximation a{x) has the properties that 
a(x c ) = fki(xc) and Vo(jv) = V fhi(x c ). Zero- order or 
higher-order corrections are easily constructed as well. 

Because the /^-correction can make any two unrelated 
functions match to first order, the framework admits a 
wide range of models. In the worst case of performance, 
the subproblem will yield a good predictive step .s c for the 


high-fidelity model only at the point x c . Thus, the high- 
fidelity information may potentially have to be computed 
at every step to re-calibrate the low-fidelity information. 
This would lead AMMF to become, at worst, a conven- 
tional optimization algorithm. At its best, the AMMF 
would be able to take many steps with the corrected low- 
fidelity model before resorting to re-calibration with ex- 
pensive evaluations. Which scenario actually takes place 
depends on the problem at hand. 

AMMF has shown promise with low-fidelity models 
represented by data-fitting approximations and variable- 
resolution models. In an attempt to evaluate the poten- 
tial worst-case scenario, we are now considering manag- 
ing variable physical fidelity models. We view this model 
combination as the potential worst-case scenario, because 
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Figure 3: Mach number contours for viscous vs. inviscid model 




Figure 4: Drag coefficient contours of the viscous and inviscid models 


low- fidelity physics models are expected not to capture 
the behavior of the high-fidelity counterparts accurately, 
or at all. over some or all regions of interest in the design 
space. 

Demonstration problem 

We consider aerodynamic optimization of a two- 
element airfoil designed to operate in transonic condi- 
tions ( 1 8|. The inclusion of viscous effects is very impor- 
tant for obtaining physically correct results. Therefore, 
the high-fidelity model will be the Reynolds-averaged 
Navier-Stokes equations and the low-fidelity model will 


be the Euler equations. The flow solver, FUN2D, used 
for this study follows the unstructured mesh methodol- 
ogy [ 1 9]. Sensitivity derivatives are provided via a hand- 
coded adjoint approach [20]. 

The mesh for the viscous model depicted in Fig. I 
consists of 10440 nodes and 20900 triangles. The mesh 
for the inviscid model, shown in Fig. 2, comprises 1947 
nodes and 389(> triangles. The Mach number is = 
0.75, the Reynolds number is IU — 9 x KF, the global 
angle of attack is o = 1°. 

Fig. 3 depicts the Mach number contours for the vis- 
cous and inviscid model, respectively. The boundary and 
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Test 

hi-fi eval 

lo-fi eval 

total time 

factor 

PORT with hi-fi surrogates, 2 var 

15/15 


negligible 


AMMF with surrogates, 2 var 

3/3 

18/9 

negligible 

% 5 

PORT with hi-fi analyses, 2 var 

14/13 


% 1 *2 hrs 


AMMF with direct analyses, 2 var 

3/3 

19/9 

« 2.1 1 hrs 

^ 1.98 


Table I : AMMF performance vs PORT 


shear layers are clearly visible in the viscous case. Be- 
cause of the importance of the viscous effects in this prob- 
lems, the use of the inviscid equations for the low-fidelity 
model should present an important test for the present ap- 
proach. 

The objective of this problem is simply to minimize 
the drag coefficient by adjusting the global angle of at- 
tack and the y-displacement of the flap. In this study, 
we restrict ourselves to two design variables to enable vi- 
sualization. The baseline case for both models was con- 
structed at a = 1° and zero //-displacement of the flap. 

Fig. 4 depicts the level sets of the drag coefficient 
for the viscous and inviscid models. The problem ap- 
pears to support the worse-case scenario: not only is the 
low-fidelity model not a good representation of the high- 
fidelity model but, in addition, the descent trends in the 
two models are reversed. The solution for each problem 
is marked with a circle. Thus the problem provides a good 
test of the methodology indeed. 

The computational expense necessary to calculate 
functions and derivatives in the viscous case is consid- 
erably greater than that for the inviscid model. We con- 
ducted our experiments on an SGI™ Origin™ 2000 work- 
station with four MIPS RISC R 10000 processors. One 
low-fidelity analysis took approximately 23 seconds and 
one low-fidelity sensitivity analysis took between 70 and 
100 seconds. In contrast, one high-fidelity analysis took 
approximately 21 minutes and one high-fidelity sensitiv- 
ity analysis took between 21 and 42 minutes to compute. 
The measures were taken in CPU time. Thus, the time 
per low-fidelity evaluation may be considered negligible 
compared to that required for a high-fidelity evaluation. 

Numerical results 

We conducted the following computational experi- 
ments. Because our test problem has expensive function 
evaluations, we first built spline substitutes both for the 
viscous and the inviscid model. Error analysis indicated 
that the spline fit was highly satisfactory for both mod- 
els. It should be emphasized that we did not use these 


substitutes in the conventional sense, i.e., they were not 
used to provide lower- fidelity models. Instead, they sim- 
ply served to provide low-cost substitutes for both mod- 
els for the problem components in the testing phase. Of 
course, such a test would never be conducted in a non- 
research setting, nor would it be considered for a problem 
with more than a few variables. In our setting, however, it 
saved us much time by providing an excellent approxima- 
tion of the actual functions with respect to descent charac- 
teristics at a tiny fraction of computational cost. After we 
ascertained the correctness of our procedures, tests were 
conducted directly with the flow and adjoint solver, with- 
out recourse to substitutes, because the substitutes were 
expected to smooth out the problem to a certain degree. 

The problems were first solved with single-fidelity 
models alone by using well-known commercial optimiza- 
tion software^ FORT [21], in order to obtain a baseline 
number of function evaluations or iterations to find an 
optimum. The problems were then solved with AMMF. 
Identical experiments were conducted with spline substi- 
tutes and with the actual flow and adjoint solver. 

For each experiment, performance of AMMF was 
evaluated in terms of the absolute number of calls to the 
high and low-fidelity function and sensitivity calculations. 
Because the time for low-fidelity computations was neg- 
ligible in comparison to the high-fidelity computations, 
we estimated the savings strictly in terms of high-fidelity 
evaluations. Table 1 summarizes the number of function 
(first number) and derivative (second number) computa- 
tions expended in PORT and in AMMF. 

Given the dissimilarity between the high-fidelity and 
low-fidelity model, we were initially surprised to find 
that the AMMF performed well: it consistently yielded 
approximately fivefold savings in terms of high-fidelity 
computations. The result held both for the spline substi- 
tutes and the actual functions. Optimization applied to 
both cases produced nearly identical iterates. 

Following an analysis of the results, we concluded that 
the savings were not surprising after all. For our combi- 
nation of models, the /^-correction worked extremely well. 
This is illustrated in Fig. 5. The plot on the left shows the 


^The use of names of commercial software in this paper is for accurate reporting and does not constitute an official endorsement, either ex- 
pressed or implied, of such products by the National Aeronautics and Space Administration or Institute for Computer Applications to Science and 
Engineering. 
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Figure 5: Drag coefficient level sets of the viscous and corrected inviscid models 


level sets of the high-fidelity model with the solution. The 
plot on the right depicts the level sets of the low-fidelity 
model ^-corrected at the initial point. The initial point is 
marked hy a square. We note that the correction is not 
applied to the entire feasible region during iterations of 
optimization algorithm. Here, we applied the correction 
to the entire region to visualize the affect of the correction 
on the low-fidelity function. The figure clearly shows that 
the correction, using the function and derivative informa- 
tion at the anchor point (at this iteration - the initial point), 
reversed the trend of the low- fidelity model, allowing the 
optimizer to find the next iterate in the left upper corner 
of the plot, marked by a circle. Similar analysis can be 
conducted for all iterations. In fact, AMMF located the 
solution (a = 1.(5305°, flap //-displacement = -0.0048) 
of the high-fidelity problem already at the next iteration. 
The high-fidelity drag coefficient at the initial point was 
f 'mitial _ () .()l 7 i, the high-fidelity drag coefficient at 

the solution was ( f j] na ^ = 0.0 118, a decrease of approxi- 
mately 13.45%. 

Concluding remarks 

We believe that the results obtained in this study with 
AMMF and variable- fidelity physics models arc promis- 
ing. We observed fivefold savings in terms of high-fidelity 
evaluations compared to conventional optimization. De- 
spite the great dissimilarity between the models, AMMF 
was able to capture the descent behavior of the high- 
fidelity model with the assistance of the first-order cor- 
rection. When the models are greatly dissimilar, first- 
order information appears indispensable in obtaining rea- 
sonable descent directions. 


Given the present results, we are cautiously optimistic 
about several much larger test cases (c.g.. 84 variables) 
that are currently under investigation. Large problems 
must be tested carefully in AMMF in order to ascertain 
that its performance is not in some measure an artifact of 
the problem dimensionality. This does not appear to be 
the case, because AMMF was previously tested on prob- 
lems with over ten variables. However, the tests currently 
conducted with realistic physical models should prove 
more conclusive, regardless of the outcome. 

The performance of AMMF with variable-fidelity 
physics models raises a number of intriguing questions 
about the nature of the corrections and an optimal choice 
of low-fidelity models for a large set of problems. These 
questions are currently under investigation. 
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